ABSTRACT 

In one embodiment, a video encoder encodes digital video using motion 
segmentation. According to one embodiment, motion segmentation is performed 
using multiple frames. The motion segmentation may operate by determining 
multiple classification hypotheses and by re-classifying poorly classified regions 
according to the multi-frame hypothesis tracking algorithm. This involves 
determining a similarity measure for each hypothesis class, and then assigning a 
classification to the region with the hypothesis class that is most similar or consistent 
with past or future data. This hypothesis tracking algorithm for re-classification can 
handle inherent problems like occlusion and ambiguous motion classification, that 
plague standard methods. In another embodiment, the motion segmentation algorithm 
may be integrated into a larger video encoding system. The system may incorporate 
motion estimation, motion classification, identification and selection of poor blocks, 
and re-classification of poor blocks using the multi-frame hypothesis tracking 
algorithm. 
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